Frame interpolation and bidirectional prediction of video using compactly encoded optical-flow fields and label fields

We consider the problems of motion-compensated frame interpolation (MCFI) and bidirectional prediction in a video coding environment. These applications generally require good motion estimates at the decoder. In this paper, we use a multiscale optical-ow-based motion estimator that provides smooth, natural motion elds under bit-rate constraints. These motion estimates scale well with change in temporal resolution and provide considerable exibility in the design and operation of coders and decoders. In the MCFI application, this estimator provides excellent interpolated frames that are superior to conventional motion estimators, both visually and in terms of PSNR. We also consider the eeect of occlusions in the bidirectional prediction application, and introduce a dense label eld that complements our motion estimator. This label eld enables us to adaptively weight the forward and backward predictions, and gives us substantial visual and PSNR improvements in the covered/uncovered regions of the sequence.